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Abstract 

We consider the problem of collaborative inference in a sensor network with heterogeneous and statistically 
dependent sensor observations. Each sensor aims to maximize its inference performance by forming a coalition 
with other sensors and sharing information within the coalition. It is proved that the inference performance is a 
nondecreasing function of the coalition size. However, in an energy constrained network, the energy consumption 
of inter-sensor communication also increases with increasing coalition size, which discourages the formation of the 
grand coalition (the set of all sensors). In this paper, the formation of non-overlapping coalitions with statistically 
dependent sensors is investigated under a specific communication constraint. We apply a game theoretical approach to 
fully explore and utilize the information contained in the spatial dependence among sensors to maximize individual 
sensor performance. Before formulating the distributed inference problem as a coalition formation game, we first 
quantify the gain and loss in forming a coalition by introducing the concepts of diversity gain and redundancy loss 
for both estimation and detection problems. These definitions, enabled by the statistical theory of copulas, allow us to 
characterize the influence of statistical dependence among sensor observations on inference performance. An iterative 
algorithm based on merge-and-split operations is proposed for the solution and the stability of the proposed algorithm 
is analyzed. Numerical results are provided to demonstrate the superiority of our proposed game theoretical approach. 


Index Terms 

Wireless sensor network, Distributed inference, Fisher information, Kullback-Leibler divergence. Game theory, 
inter-modality dependence. Copula theory 


I. INTRODUCTION 

In a distributed inference problem, each sensor collects observations regarding a phenomenon of interest, then 
shares them with other sensors or transmits them to the fusion center (FC). To reduce the energy cost for communi¬ 
cation, the observations may be processed before transmission. The distributed nature of wireless sensor networks 
induces a tradeoff between minimizing the communication cost and maintaining acceptable performance levels. 
Although there has been a lot of work on distributed inference, including distributed detection and distributed 
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estimation, with conditionally independent observations, much less has been done for the case of dependent 
observations Q-inni. 

The spatial correlation among the sensor observations is a significant characteristic which can be exploited 
to significantly enhance the overall network performance, including inference performance and energy efficiency. 
Typical applications of Wireless Sensor Networks (WSNs) require spatially dense sensor deployment in order to 
achieve satisfactory coverage. As a result, proximal sensors recording information about a single event are highly 
correlated with the degree of correlation increasing with decreasing internode separation. Such dependence among 
adjacent sensors or agents also exists in other intelligence aggregation networks. For example, in a crowd sourcing 
network, agents with the same backgrounds or having active interactions (e.g., following each other on social 
websites) are likely to have correlated knowledge/observations about the same event. Any network consisting of 
dependent agents having the ability to take measurement of the environment and making inference based on available 
observations, such as wireless sensor networks, cognitive radio networks or a crowd sourcing network, are within 
the consideration of this work. For the simplicity of presentation, we use the term “sensor” to represent an intelligent 
agent, which can be a real sensor, a cognitive radio, or a participating agent in a crowd sourcing network, in the 
remainder of this paper. Dependence among observations may make some sensors’ observations redundant. An 
extreme case is when two sensors’ observations are completely positively correlated, one of the two sensor will 
become “redundant”. Since transmitting “redundant” observations from battery powered sensors to remotely located 
FC is energy inefficient, we have an opportunity to conserve energy via local collaboration in a densely located 
sensor network. 

The effect of dependent noise and hence dependent observations on Fisher Information (FI) has been studied 
by Yoon and Sompolinsky in M- The authors showed that, in the biologically relevant regime of parameters, 
positive correlations degrade estimation performance compared with an uncorrelated population. Sundaresan et al. B 
considered location estimation of a random signal source where they focused on improving system performance 
by exploiting the spatial dependence of sensor observations. Parameter estimation with dependent observations in a 
variety of communication scenarios was considered in na, but was limited to the case of “geometric” dependent 
Gaussian noise. 

Different approaches have been employed to study the detection problem with correlated observations, most of 
which focus on small sample size m, M- It has been shown that correlation degrades overall performance either 
in the case of a binary signal in equicorrelated Gaussian noise ca or in the cases where correlation increases 
with the decrease in inter-sensor distance na. In parallel sensor networks, the fusion of statistically dependent 
observations is considered under various scenarios 0-181, na, M and the design of local decision rules is 
investigated in a through the introduction of hierarchical independence model. Performance of WSNs exposed to 
correlated observations is also assessed using the theory of large deviations 0. 

In this paper, we formulate a novel distributed inference framework which fully exploits and utilizes the inter¬ 
sensor dependence for improved overall system performance, given the inherent tradeoff between inference per¬ 
formance and transmission efficiency. This framework provides an opportunity to maintain a comparable inference 
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performance to that of a centralized framework while achieving greater transmission efficiency, in networks with 
correlated sensors. In such a framework, there is no FC and each individual sensor is capable of sensing and 
computing. Sensors form non-overlapping coalitions and collaborate by sharing their observations within a coalition. 
In the process of forming coalitions, each sensor selfishly aims to maximize its own inference performance, and 
thus the performance of the coalition to which it belongs, as will be evident later. The problem is to find a set of 
non-overlapping coalitions such that each sensor’s inference performance is maximized under certain energy cost 
constraints. To model and analyze the spatial dependence among sensor observations which might be heterogeneous 
(different marginal distributions), we use copula theory, which has been applied for inference with dependent 
observations in ||5l-ll8l. 

In our framework, each sensor is characterized not only by its individual inference performance achieved with its 
own observations, but also by its dependence with other sensors in the network. Unlike the individual performance 
which is fixed and unchangeable no matter which coalition the sensor belongs to, its dependence with other sensors 
plays different roles in different coalitions. In order to quantify the gain and loss of collaboration resulting from inter¬ 
sensor dependence, we introduced the concepts of redundancy loss and diversity gain for the distributed estimation in 
El. Other definitions of diversity are available in different contexts in the signal processing literature. In cognitive 
radio systems, diversity is acknowledged as the benefit of collaborative sensing and diversity order in various 
collaborative spectrum sensing schemes is quantitatively determined in M- In communication systems, diversity is 
widely adopted as an indicator of the signal-to-noise ratio (SNR) dependent behavior of inference performance based 
on multiple received signals mil, EOl. In the distributed inference problem that we are considering in this paper, 
diversity gain quantifies the positive effect of dependence on inference performance, in contrast with redundancy 
loss, which quantifies the redundant information induced by the dependence among sensor data. 

Since an optimal solution to the coalition formation problem may not exist, namely there may not be such a 
partition that every sensor’s performance is maximize, our best hope is to find a stable solution. Thus, we use 
game theoretical approach and formulate our collaborative distributed inference problem as a coalition formation 
game. Game theory has been widely applied to statistical inference, such as measurement allocation for localization 
ED, communication networks ED, and spectrum sensing ES- An iterative algorithm based on merge and split 
operations ll24ll is proposed in the literature to find a stable solution for the coalitional games discussed above. 

Building on our preliminary work in ifTTll . which focused on an estimation problem, in this paper, we study the 
general problem of distributed inference in sensor networks with local collaboration. The major contributions of 
this paper can be summarized as follows: 

• We fully investigate the different roles played by inter-sensor dependence for inference problems, including 
both detection and estimation; and we define diversity gain and redundancy loss to respectively characterize 
the benefit and loss in forming coalitions due to inter-sensor dependence in this more generalized setting. 

• We formulate a coalition formation game for the more generalized distributed inference problem with dependent 
observations for large heterogeneous sensor networks. We design an iterative algorithm based on merge and 
split operations to solve the coalition formation game, which is more efficient than other approaches available 
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in the existing literature. 

The rest of the paper is organized as follows. Basic concepts of copula theory and coalitional games are introduced 
in Section |II] as background knowledge. Section |III] describes the system model and the inference problem is 
formulated in general. Section |IV] introduces the distributed estimation problem and analyzes the role of inter¬ 
sensor dependence. Section [V] analyzes the problem of distributed detection and quantifies the dependence-related 
diversity gain and redundancy loss. Section |Vl] proposes a coalition formation game and a merge-and-split based 
algorithm to obtain a stable solution. Section Ivnl presents and discusses simulation results. We provide concluding 
remarks in Section [Vllll 


II. Preliminaries 

A. Copula Theory 

Simply put, copula functions couple multivariate joint distribution functions to their component marginal distri¬ 
bution functions ||25l. We begin with the definition of a copula function. 

Definition 1. A function C : [0,1]'^ —> [0,1] is an N-dimensional copula if C is a joint cumulative distribution 
function (CDF) of an N-dimensional random vector on the unit cube [0,1]^ with uniform marginals 425I/ - 427I/ . 

The application of copulas to statistical signal processing is made possible largely because of the following 
theorem by Sklar ll25l . 


Theorem 1 (Sklar’s Thoerem). Consider an N-dimensional distribution function F with marginal distribution 
functions Tj,... , Fj^. Then there exists a copula C, such that for all Xi, . .., xjv m [~oo, oo] 

F{xi,X2,...,Xn) = C{Fi{xi), F2 {x2), ■.., Fn{xn)) (1) 

If Fn is continuous for 1 < n < N, then C is unique. 


Conversely, given a copula C and univariate CDFs Fi,..., Fn, F as defined in (HJ is a valid multivariate CDF 
with marginals Fi,... ,Fn. According to Sklar’s Theorem ll25l . for continuous distributions, the joint probability 
density function (PDF) can be obtained by differentiating both sides of O 


f{Xi,...,XN) = fn{Xn)'^ c(fi (li),..., Fjv (xat ) | (/)) 


( 2 ) 


(3) 


where c(-) is termed as the copula density function and is given by 

_ d^C{ui,...,UN) 

^ dui, ..., duN 

with Un = Fn{xn). Copula functions contain a dependence parameter cj) that quantifies the amount of dependence 
among the N random variables. It needs to be noted that this is well-suited for modeling heterogeneous random 
vectors where a different distribution might be needed to model each marginal a;„. Several copula functions are 
defined in the literature ll25l of which the elliptical and Archimedean copulas are widely used. 
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An attractive feature of copulas is their relationship with the nonparametric rank-based measures of dependence, 
such as Kendall’s rQ. The relationship for a copula C, and the Kendall’s r for random variables X and Y is given 
by 1251 p. 159] 

Tx,v = 4 J j C{u,v)dC{u,v) — 1 (4) 

where u = Fx{x), v = Fy(y). The relationship in (l4]i results in a one-to-one correspondence between Kendall’s t 
and copula parameter (p, based on which a rank-based estimation of dependence parameter cp can be performed. 

B. Coalitional Game Theory 

To facilitate the formulation of our problem, we introduce basic concepts in coalitional game theory. Let J\f = 
{1,2,..., TV} be a set of fixed players called the grand coalition. Nonempty subsets of Af are called coalitions. A 
collection (in the grand coalition Af) is any family S := (S'!,..., of mutually disjoint coalitions. If additionally 
G^iSj = Af, the collection S is called a partition of AA. 

Assuming a comparison relation >, TZ = {i?i,..., Rk}>S = {5i,..., means that the way TZ partitions AA, 
where AA = is preferred over the way S partitions AA based on some performance measure. 

Pareto order can be used as a comparison relation o. For a collection TZ = {i?i,..., Rk}, the utility of a player j 
in a coalition Rj G TZ is denoted by ^j{TZ), and the Pareto order is defined as follows 

TZ>S ^ {$,(7^) > $,(5),Vj e 7^,5} (5) 

with at least one strict inequality for a player k. 

Apt and Witzel ll24l proposed an abstract approach to coalition formation that focuses on simple merge-and-split 
rules to transform partitions of a group of players. Details of coalition formation will be introduced in detail in 
Section HV] 


III. SYSTEM MODEL 

We consider a physical phenomenon being continuously observed by a set of densely deployed sensors, which 
is represented by AA = {1,2,...,A^}. Each sensor’s observation is Xn- Let 9 be the parameter that denotes the 
phenomenon of interest in the received signal Xn at sensor n for the general inference problem. When we consider 
a detection problem, 9 represents a binary discrete variable, while in the case of parameter estimation, 0 is a 
realization of a continuous random variable 0 with PDF fe{‘)- Due to high density of sensors in the network 
topology, sensor observations are highly correlated spatially. 

In a non-collaborative setting, each sensor continuously senses the environment, and locally makes inference 
about the unknown parameter 9 solely based on its own observations. In this work, we consider a collaborative 

* Let (JYi, Yi) and (X2, Y2) be two independent pairs of random variables with a common joint distribution function H and copula C, 
The population version tx,y of Kendall’s r is defined as the probability of concordance minus the probability of discordance: tx,y = 
P[iXi - X2 ){Yi - y2) > 0] - P[ixi - X2 ){Yi - Y 2 ) < 0]). 
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setting where collaboration exists within coalitions. Participating sensors are required to act in accordance with the 
following rules: 

1) Sensors first form coalitions, and each sensor can only join one coalition. 

2) Once the coalitions are formed, a sensor can request observations from all the other sensors in the same 
coalition and make an inference; it also has to transmit its observations to the other collaborating sensors 
upon their request. 

In such a collaborative setting, each sensor, as an independent agent, aims to improve its own inference performance 
through collaboration with the most “useful” sensors. The coalition formation process, namely, how the coalitions 
should be formed such that each selfish sensor has its performance maximized, is the focus in this paper. 

An intuitive solution would be that all the sensors form a grand coalition such that every sensor enjoys the 
benefit of collaboration to the maximum extent. However, in an energy constrained network, each sensor’s energy 
is finite and a communication cost is incurred when it transmits its observations to collaborating sensors. Let r 
be the average number of requests initiated by each sensor in the network per unit time interval. Then, for any 
sensor in coalition S, the number of requests that have to be responded to within a unit time interval is r(|5'| — 1), 
where l^l denotes the cardinality of coalition S. We assume that energy consumption for a single transmission is 
Et- The average energy consumption per unit time interval for each sensor in coalition S is E{S) = r(|S'| — l)Et, 
which increases as the coalition size increases. Let the energy consumption of a coalition be the average energy 
consumption per sensor in this coalition, which is the same quantity E{S) = rd^j — l)Et. Thus, from the point 
of view of energy consumption, smaller coalitions are preferred. In order to guarantee adequate sensors’ lifetime, 
we enforce the energy consumption constraint as follows 

E{S) = r(|S'| - l)Et <a, VS' € S. (6) 

Then the problem is to find the optimal partition S of the set of sensors Af such that each sensor’s inference 
performance is maximized subject to the energy constraint in (|6]l. 

maxA„(S), Vn € J\f 

Sev 

subject to E{S) < a, WS € S (7) 

where A„(S) represents the inference performance of sensor n under partition S, and V denotes the set of all 
possible partitions of Af. 

For the optimization problem in (I7]i, an exhaustive approach in which we search over all possible partitions will 
invoke a very high computational complexity. According to ll28l . for a network with N sensors, the total number 
of partitions is 0{N^). Besides computational issues, an exhaustive search may not be able to give us a solution 
to the problem in (|7]i, since there may not exist a partition such that each sensor’s performance is maximized 
simultaneously while the energy consumption constraint is satisfied. For the same reason, if each sensor solves its 
optimization problem iteratively by itself, the overall system optimization algorithm may not converge. Thus, our 
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best hope is to find a stable solution and to do that we use a game theoretical approach. Before formulating 
the distributed inference problem as a coalition formation game, we need to define and analyze the gain and the 
loss of each sensor when it joins a coalition, in the context of dependent observations. The analysis is carried out 
respectively for the problem of estimation and detection in the following two sections. 


IV. Collaborative Distributed Estimation 


In the estimation problem, the optimization problem can be formulated as the minimization of Posterior Cramer- 
Rao Lower Bound (PCRLB), or equivalently, the maximization of posterior Fisher Information (FI), which is given 
by 


FI = —Ex,e 


92 


log/x(x;6») 


= -]Ex,0 
-E 

= I + Ip 


902 
log/x(x; 9) 


902 

log/e(6') 


( 8 ) 


where /x represents the joint PDF of X := [Xi,... ,Xjv]; I and Ip represent the sensor data’s contribution and 
prior’s contribution to posterior FI respectively. The prior’s contribution is fixed given the distribution of 0. Thus, 
we only consider sensor data’s contribution. Since / is the FI averaged over the distribution of 0, it is referred to 
as the average FI ll2^ . For the coalition S whose set of observations is xs := [xn,Vn G S'], the average FI it can 
achieve is given as 


I(S) = -E 


log/xs(xs;^) ' 

902 


(9) 


where /xg (•) denotes the joint distribution of Xs and the expectation is taken with respect to the joint distribution 


of Xs,0. 


Remark 2. As an immediate result of the modus operandi of the network, the estimation performance, i.e., average 
FI, achievable at sensor n that is in coalition S, denoted by In{S), equals the average FI contained in coalition 
S, which is denoted by I{S). That is 

US)=I{S), VnGS 


Proposition 1. I{S) is a nondecreasing function of the cardinality of S. 


^A .stable solution can simply be interpreted as a partition where no player has the incentive to leave the current partition. Stability will be 


discussed in detail later in this work. 
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Proof: We need to show that I{S) > I{S ), for S' C S'. According to the definition of average FI of coalition 
S in dH) 

52 


/(S) = -E 

= -IE; 

Eo' 


■log/xs(xs;0) 


g02 log/x^,(xsS0) 


+ 


®S\S' |S' [ g02 ■f^s\s' ^^S\s' l^s'! ^)] 


( 10 ) 


where S \ S denotes the relative complement of S with respect to S, i.e., {n : n G S, n ^ S }. It can be noted 
that the first term in (fTOl i corresponds to the average FI of S , and the second term is the expected conditional 
average FI of S \ S . Due to the non-negativity of conditional FI, we have 


I{S) 


I{s') + Es, 


I{S\S'\S') 


> I{S') 


( 11 ) 


Remark 3. When the transmission cost is assumed to be zero, i.e., Et = 0, a grand coalition forms. It is proved 
in Proposition Q] that average FI does not decrease by including more sensors in a coalition. Thus, if there is no 
communication cost, all the sensors will collaborate for a better estimation performance. 

It is clear from Proposition [T] and the definition of E{S) = r(|S| — l)Et that, as the coalition size increases, both 
estimation performance in terms of I{S) and the energy consumption E{S) increase with it. There is a tradeoff 
between the estimation performance and communication efficiency. Each sensor aims to maximize its estimation 
performance subject to an energy constraint. The problem is formulated as the following; 

max/ji(iS), Vn G Af 

S(ZV 

s.t. r{\S\ - l)Et <a, VS e S (12) 

where In{S) represents the average FI of sensor n under partition S, i.e. /„(5) = IniS), for n G S and S G S. 

A. Diversity gain & Redundancy loss 

To analyze the effect of inter-sensor dependence on the average FI for coalition S, we express the joint PDF 
of observations of sensors in coalition S in terms of the marginal PDFs and copula density function Cg, as in (|2]i. 
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Fig. 1 . GAFI corresponding to Gaussian copula vs. correlation coefficient p. (The marginal distributions are Gaussian. Identical marginals 
imply that the mai'ginal distributions ai'e the same, and heterogeneous marginals imply that the mai'ginal distributions are different.) 


using copula theory. When logcs(-; 0, </>) is twice differentiable with respect to 6, I{S) can be written as 

log (llnGS 0)csi-;9, </>)) 


I{S) = -E 


nGS 

nGS 


92 log cs(-;6>» 




(13) 


where represents the average FI achieved by a single sensor n in a non-collaborative setting, and Ic{S) represents 
the FI that is induced by the dependence structure cs- Thus, the average FI for a coalition S can be written as 
the summation of average FIs of each individual sensors in S and Ic{S). We call Ic{S) the generalized average 
FI (GAFI) for the copula density function cs because it may not satisfy the non-negativity property of average FI. 
Figure [T] shows the GAFI for a Gaussian copula as a function of the dependence parameter p. It is shown that 
for the case of identical marginal distributions, GAFI is nonpositive and decreases with an increase in p. More 
complicated behavior of GAFI is observed when marginal distributions are different as seen in Figure [T] 

The following proposition provides some insights into the properties of the GAFI for a two-sensor coalition. We 
assume the joint distribution to be bivariate Gaussian which can be written as a product of Gaussian marginals and 
a Gaussian copula. 


Proposition 2. Let the random vector [X, be bivariate Gaussian distributed, i.e., [V, ~ N{ij,,Yixy), 

where fi = [pxid), Py{0)]^, 


Exr = 


f 


^x ^xc^yPxy 
\aY<7xPYX tTy / 




and 6 is the parameter to be estimated (Without loss of generality, let ^ 


< 1, where the derivatives are 


April 15, 2015 


DRAFT 









































10 


taken with respect to 9), then we have: 

1) Ic{X,Y), the GAFI of copula Cxy> is convex function of pxY minp^y Ic{X,Y) = is reached 

„ _ <^x Uy{S) . 

af PXF - 

2) Ic(X,Y) < 0 when pxY lies between 0 and ■ 

3) When = I Ic{X,Y) > 0 for p G [-1,0] and Ic{X,Y) < 0 for p G (0,1]. Furthermore, Ic{X,Y) 

is a monotonically decreasing function of pxY- 


Proof: According to the definition of GAFI in ( [T3b 

log cxY {Fx{x;9),FY{y,9)-, pxy) 


Ic{X,Y) = -E 


-1 


502 

{‘^PxY 9x{(^)9Y{(^)'^xe^Y 


Pxy) 

~p\y{Px{^)'^y +A^f(^)o'x)} 


It can be shown that 


0 


d^Ic{X,Y) 


dp 


>0, VpxFe(-l,l) 


jty 


thus, Ic{X,Y) is convex. By setting 


and knowing that 


dIeiX,Y) 

dpxY 

ax Py^^) 


we get 


P = 


cry p'xie) 
ax Py{S) 


= 0 


< 1 


OY Px{S) 
led at p* = 

By setting (HaI) equal to zero, we get two solutions: 


i^ix,Yr = -^ 


Thus, the minimum of Ic{X,Y) is obtained at p* = ^ , which is Ic{X,Y)* = — . 


(14) 


2/r'y(0)^y(0)ajCCry 
Px{^)aY F PY{9)''^a\ 

Combined with the convexity of the function, it can be concluded that Ic{X, y) < 0 when pxY £ [inin{pi, P 2 }, max{pi, P 2 Y\- 
By letting axPyi^) ~ aYPxi^) i*' G3J’ conclusions in (3) can be directly derived. ■ 

Remark 4. When pxY = 0, Ic{X,Y) = 0, meaning that the average FI of the coalition is solely the summation 
of individual average FIs of X and Y; when pxY = ^ , IciX, Y) is just the smaller individual average FI 

of the two sensors with a minus sign. In the latter case, the sensor with larger individual average FI gains nothing 
in estimation performance by collaboration. 


^The dependence of Ic{X,Y) on correlation coefficient pxY not rnade explicit for notational convenience. 
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Remark 5. In our formulation, a sensor n prefers to collaborate with sensor m with which it has a positive 
Ic{Xn, Xm) than sensor k with which it has a negative Ic{Xn, Xk), when sensor m and sensor k have identical 
individual performances in terms of average FI. This is because to sensor n, sensor m is more “valuable ” than 
sensor k in the sense that due to inter-sensor dependence, some of sensor k’s information is redundant for sensor 
n. 

Definition 2. For Ic{X,Y) < 0, we define -Ic{X,Y) to be pairwise redundancy loss denoted as Irl{X,Y), 
Otherwise we define Ic{X,Y) to be pairwise diversity gain denoted as Idg{X,Y). 

The definitions of diversity gain and redundancy loss allow for a better characterization of the different roles 
that pairwise inter-sensor dependence may play. General properties of the GAFI of multivariate copulas can be 
analyzed using vines which is a graphical method of constructing multivariate copulas Eni, 111]. The joint PDF of 
N random variables expressed in terms of a D-vine decomposition is given by: 

/x(x) = 

N N-lN-j 

n fM n n ( 15 ) 

n—l j—1 k—1 

where j = [j + 1,..., j' + fc — 1] and xj = ,..., Thus, a multivariate copula is decomposed into the 

product of bivariate conditional copulas. Therefore, Ic{S), the corresponding GAFI of the copula in any coalition 
S can be written as: 

|S|-l|S|-j 

= E E uxj,x,+,\xj) 

j=l k=l 
\S\-l\S\-j 

= EE ^dg{Xj,Xj+k\X.j)lj^(^Xj,Xj+k\Xj)>0 

j=l fe=l 
|S|-l|S|-i 

“EE ^rliXj,Xj+k\Xj)li^^^Xj,Xj+k\Xj)<0 

j=l k=l 

= IdgiS) - IrliS) (16) 

where Ipj denotes the indicator function, and Idg{S) and Iri{S) respectively represent the diversity gain and 
redundancy loss in the coalition S. Each of them is a summation of pairwise diversity gains or pairwise redundancy 
losses in coalition S. Until now, we have quantified the benefit and cost of forming a coalition S incurred by 
dependent sensor observations in the problem of distributed estimation. In the following section, the counterparts 
of diversity gain and redundancy loss for the distributed detection problem will be investigated. 

V. Collaborative distributed detection 

In the detection problem, 0 is a bi-valued variable which takes the value 6q under hypothesis Hq and takes the 
value 01 under hypothesis Hi. In this paper, we employ Kullback-Leibler Divergence (KLD) as the performance 
metric. KLD can be interpreted as the error exponent in the Neyman-Pearson framework, which means that the 


April 15, 2015 


DRAFT 


12 


probability of miss detection goes to zero exponentially with the number of observations at a rate equal to KLD. 


Thus, KLD characterizes the asymptotic detection performance. We denote KLD by D and define it as follows 



(17) 


where denotes the expectation taken with respect to the joint distribution of X under hypothesis Ha. For a 

coalition S, the detection performance it can achieve, in terms of KLD, is D{S). 



Remark 6. The KLD for sensor n, i.e., Dn{S), in a coalition S is the same for all n € S. Similar to Remark^ 
for the estimation problem, we can write 


Dn{S)=D{S), yneS 


Proposition 3. D(S) is nondecreasing in 151. 


Proof: In order to prove that D{S) does not decrease by including new members to the existing coalition, we 
need to show that for any 5 C 5, D{S ) < D{S). 



= i7(5')+Ex^,|rro [d{S\s') 

> DiS') 


(19) 


The last inequality is because of the non-negativity property of conditional KLD. 


Remark 7. A grand coalition forms when communication cost is zero, i.e., Et = 0. 

It is noted that, as |5| increases, both D{S) and E{S) increase, indicating a tradeoff between the detection 
performance and energy consumption. In our formulation, each sensor selfishly aims to maximize its own detection 
performance, i.e., the KLD that can be obtained using shared observations within the coalition to which it belongs, 
subject to an energy constraint. The problem can be formulated as the following 


maxi7„(5), 'in ^ N 
Se-P 

s.t. r(|5| - l)Et < a, V5 e 5 


( 20 ) 


where Dn{S) represents the KLD of sensor n under partition S, i.e., Dn{S) = Dn{S), for S G S and n G S . 


April 15, 2015 


DRAFT 













13 



Fig. 2. GKLD corresponding to different copulas vs. Kendall’s t: Gaussian marginals are assumed. 


A. Diversity Gain and Redundancy Loss 

The effect of inter-sensor dependence on the KLD can be analyzed by expressing the joint PDF of the observations 
of sensors in coalition S in terms of the marginal PDFs and copula density function Cg. By copula theory, the KLD 
corresponding to Xg can be written as 


D{S) = 



n In{Xn\Ho)cs{-\(t)o, Hq) 

f ( |7FT— „ s /xs(x5|go)dxs 


n^S 


— '^Dn + Exs \Ho 
uGS 


CS {Fn{Xn\Ho),yn € S\4>q,Ho) 
CS {Fn{x„\Hi),\/n G 5101,iFi) 


= Y,Dn + D,{S) 

tiGS 


( 21 ) 


where is the KLD achieved by sensor n with its own observations in a non-collaborative setting and 0^ is 
the dependence parameter of the copula density under hypothesis Hi, z = 0,1. The KLD between the two joint 
distributions of sensor observations in S under hypotheses Hq and Hi can be decomposed into two terms, as shown 
in (I 2 TI 1 . The first term represents the summation of KLDs corresponding to individual sensors in S and the second 
term Dc{S) measures the distance between the two joint distributions introduced by the dependence structure. We 
call Deis) the Generalized KLD (GKLD), because the arguments of cs{-\Hq) and cs{-\Hi) are different and thus 
violate the standard definition of KLD. In Figure |2l the GKLDs corresponding to different copulas are plotted 
against Kendall’s r. Similar trend is observed among these curves. For each curve, there exist a single t* that 
divides t G [0,1] into two intervals, each corresponding to positive or negative GKLD. Since Kendall’s t is only 
a scalar summarization of the “amount” of dependence, the behaviors of GKLDs vary for different copula models 
(structures of the dependence). 


April 15, 2015 


DRAFT 




































14 


The following proposition provides insights into the GKLD in a coalition consisting of two sensors whose 
observations follow bivariate Gaussian distribution which can be viewed as the product of two univariate Gaussian 
PDFs and a Gaussian copula. 

Proposition 4. Consider two random variables [X, ~ X([0i, 0i], Exv) under hypothesis Hi and [X, ~ 

X([0o,0o],Sxy) under hypothesis Hq, where 

I tTx (JxCrYPXY \ 

^XY = 

yJYCrxPYX CTy j 

and 9i ^ 9 q. Without loss of generality, let ax < cty, then we have: 

1) Dc{X, Y), the GKLD corresponding to the Gaussian copula cxy, is a convex function of pxY ond miup^y DdX, Y) 

— is reached at pxY = —t 

2) Dc{X, y) < 0 for pxY between 0 and 

3) For ax = <xy, Dc{X,Y) > 0 for pxY £ [~liO] Dc{X,Y) < 0 for pxY £ (0,1] and it is a monotone 
decreasing function of pxY,- 


Proof: According to the definition of GKLD, we have 

DdX,Y) 


= E 


XY\Ho 


log 


c{Fx{x\Ho),FY{y\Ho)\YxY) 


c {Fx{x\Hi), FY{y\Hi)\YxY)_ 

{9i — f 2 : ^2 , _2 \ o _ _ 1 

.2 2 n 72 7 [PxyWY F ^x) ~ ^PXYCfXCTY] 


2a\al{l-p\Y) 

where c(-|Exv) represents the Gaussian copula parameterized by 0 = Exy- It can be shown that 

d^DdX,Y) 


( 22 ) 


dp^XY 


> 0, VpxY £ (-1,1) 


Thus, the convexity is proved. By setting 


and knowing that ax < cry, we get 


dDdX,Y) 

dpxY 


= 0 


* TSIVV\* (^l-^o)^ 

P =—: DdX,Y) = -- 

CTY 2iCT-y 

combining with the convexity of the function, we know that Dc{X, Y)* is the minimum point. 

By setting (l22l i equal to zero, we get two solutions: 

2crxO'Y 

Pi - 0, P2- 2 , 2 

" F -I- ax 

knowing the convexity of the function, it can be concluded that Dc{X, y) < 0 when pxY £ [pi, ^ 2 ]- 
When ax = cry = cr, 

D.iX,Y) = 

cr^ 1 + PXY 
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It can be shown that < 0, Vpxy G (—1,1) and the sign of Dc{X, Y) is the same as that of —pxY- ■ 

Remark 8. When X and Y are independently distributed, i.e., pxY = 0, then Dc{X,Y) = 0, meaning that KLD 
is solely the summation of individual KLDs of X and Y ; when pxY = Dc{X, Y) is just the smaller individual 
KLD of the two with a minus sign. In the latter case, the sensor with larger KLD does not improve its detection 
performance by forming a coalition with the other sensor. 

Remark 9. For three sensors having dependent observations, a sensor n would prefer to collaborate with sensor 
m with which it has a positive Dc{Xn, Xm) than sensor k with which it has a negative D(.{Xm Xf), when sensor 
m and k have identical individual performance, i.e.. Dm = D^. This is because to sensor n, sensor m is more 
“valuable” than sensor k in the sense that the dependence between sensor n and m results in a larger total KLD, 
and thus contributes to a better asymptotic detection performance. 

Definition 3. For Dc{X, Y) < 0, we define -Dc{X, Y) to be pairwise redundancy loss of GKLD, denoted as 
Dri{X,Y), otherwise we define Dc{X,Y) to be pairwise diversity gain of GKLD denoted as Ddg{X,Y). 

Although the expressions of the pairwise redundancy loss and diversity gain depend on the specific problem 
that we are considering, these definitions capture the intrinsic characteristics of a sensor network with dependent 
observations and quantify the impact of the dependence in collaboration. 

According to (fTSI l. a multivariate copula is decomposed into the product of bivariate conditional copulas. There¬ 
fore, Dc{S), the GKLD introduced by the copula in any coalition S can be written as: 

\S\-l\S\-3 

E E D,iXj,X,+k\Xj) 

j=i k=i 
\S\-l\S\-j 

= E E ^dg(^Y^j+k\^j)'^D„{Xj,Xj + k\yij)>0 

j=l k=l 
\S\-l\S\-j 

“EE ^ri{Xj,Xj+k\Xj)lij^(^Xj,Xj+k\:s.j)<o 

j=i k=i 

= DdgiS) - DriiS) ( 23 ) 

Ddg{S) represents the diversity gain in the coalition S and Dri{S) represents the amount of redundant information 
included in coalition S. By noting that Ddg{S) and Dri{S) are nonnegative and nondecreasing function of [S'!, 
we can view Ddg{S) together with '^^e gain of forming S, while Dri{S) as the cost, along with 

the communication cost E{S). In the following section, a coalition formation game for distributed inference is 
formulated based on the quantification of dependence-based diversity gain and redundancy loss. 

VI. Game Formulation and Properties 

We propose a coalitional game defined by the pair {Af, V) to model our collaborative inference problem, where 
ff is the set of players (all sensors) and V is a mapping such that for every coalition S, V (S') is a closed convex 
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subset of that contains the payoffs that players in S can achieve. In order to present a generalized game 
theoretical approach to the distributed inference problem, we use a unified notation A to represent the average FI 
in the estimation problem and the KLD in the detection problem. We define the value of a coalition v{S), as an 
increasing function of the gain + ^dg{S) and a decreasing function of the costs Ari{S), and E{S): 


v{S) 


A„ + Adg{S) 

-uGS 


[Arl{S)+C{S)] 


(24) 


where C{S) is a function of the energy consumption E{S). It captures the tradeoff between inference performance 
and the energy consumption. There are certain properties that a well designed cost function C{S) should satisfy, 
here we use the logarithmic barrier penalty function given in 

-l/Mog(l-^) if EiS)< a 
+c» otherwise 

where a is the constraint on E{S), and f is a control parameter. The above cost function is an increasing function 
of E{S) for E{S) < a, while it goes to infinity when E{S) > a. Through the cost function in (l25T l, the constraint 
that E{S) < a is enforced, since for the coalitions that do not satisfy this constraint, the utility v{S) is —oo. 



Proposition 5. The payoff for each sensor in coalition S is equal to the utility of the coalition, i.e., 4>„(S') = 
v{S),Vn € S, where 4>„(>S') denotes the payoff of sensor n in the coalition S. 

Proof: The value of a coalition S defined in (l24l) is a function of its inference performance and its average 
energy consumption E{S). According to Remarks |2] and |6] the average FI or KLD for every sensor in S is given 
by the average FI and KLD of the coalition. And it is known that transmission cost E{S) of every sensor in S is 
the average transmission cost of the coalition. Hence, the coalition value v{S) is also the payoff of each player in 
it. ■ 

Now, we have a nontransferable utility coalitional game where V{S) is a singleton set (hence closed 

and convex) 

V{S) := = v{S)ffn G S} (26) 

A distributed algorithm for the above coalitions formation game among sensors is described next. 


A. Coalition formation algorithm 

For autonomous coalition formation, we propose a distributed algorithm based on two simple rules called merge 
and split m that allow us to modify a partition S of the set Af. 

Merge Rule; Merge any set of coalitions {S'!,..., Sm}, where {U^;i5'j}[>{5'i,..., S'™}, therefore, {^i,..., 

Split Rule: Split any coalition where {5i,..., 5^} > thus —)■ {^i,..., 5^}- 

Remark 10. Every iteration of the merge and split rules terminates. 
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Let us assume that the dependence information is known at the local sensors, and they autonomously form 
coalitions through merge and split operations. Let the initial partition be 5 = {S'!,..., 

repeat 

TZ — Merge(5): coalitions in S merge according to the merge rule, until no further merge occurs 
S = Split(7^): coalitions in TZ split according to the split rule, until no further split occurs, 
until No merge or split occurs 

Merge operations are first applied. Given an initial partition S = {/S'!,..., /S'm}, suppose Si seeks to collaborate 
with 82 - If the condition for merge is satisfied, a new coalition := S*! U 5'2 is formed, otherwise. Si '.= Si and 
Si attempts to merge with another coalition who shares a mutual benefit in merging. The algorithm is repeated for 
the remaining Si until all the coalitions have made their merge decisions. The resulting partition TZ is then subject 
to a split process in a similar way. Then, successive merge-and-split processes go on until the iterations terminate. 
The stability of this resulting network structure can be investigated using the concept of a defection function 

D El, El- 


Definition 4. A defection function D is a function which is associated with each partition T. A partition T = 
{Ti,... ,Tm} is D-stable if no group of players is interested in leaving 'T when the players who leave can only 
form the coalition allowed by D(7^). 

A partition T = {Ti,..., T^} of Af is D/ip-stable, if no players in T are interested in leaving T through merge- 
and-split to form other partitions in Af. A partition T is Dc-stable, if no players in T are interested in leaving T 
through any operation to form other collections in J\f El- 

D/jp-stable can be thought of as a state of equilibrium where no coalitions have an incentive to pursue coalition 
formation through merge or split. The following theorem has been proved in ll3l . 

Theorem 11. A partition is DhpStable if and only if it is the outcome of iterating the merge-and-split rules. 

Remark 12. For the proposed (Af, V) collaborative distributed inference game, the proposed merge-and-split 
algorithm converges to a lAhp-stable partition. 

It is known that if T is Dc-stable, then T is the outcome of every iteration of the merge-and-split rules and it 
is a unique Dc-stable partition El- Nonetheless, a Dc-stable partition does not always exist. A Dc-stable partition 
is not guaranteed for our collaborative game and its existence depends on the specific characteristics of the sensor 
network and the parameters of the cost function in El- 

Remark 13. For the proposed (Af, V) collaborative distributed inference game, the proposed merge-and-split 
algorithm converges to the optimal Dc-ifafe/e partition, if such a partition exists. Otherwise, the proposed algorithm 
converges to a Hhp-stable partition. 

Proof: By the properties of Dc-stable partition shown in EU. Dc-stable partition is a unique outcome of 
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any arbitrary merge-and-split iteration. Thus, if a Dc-stable partition exists, the merge-and-split algorithm finally 
converges to it ll22l . ■ 


VII. SIMULATION RESULTS 

In this section, we present the simulation results of our proposed game theoretical approach to the collaborative 
distributed inference problem. We consider a wireless sensor network with N sensors deployed in a [0,1.5] x [0,1.5] 
square area of interest. Let the location of sensor n be denoted by Sn = [s„i, 5 ^ 2 ] ■ The amount of dependence 
measured in terms of Kendall’s t between any two sensors n and m follows the power exponential model 041 

T{dn,m) =e-<^, (27) 

where = ||sn — 8^]] is the distance between nodes n and m respectively located at coordinates s„ and s^- 
We first consider a 8-sensor network where each sensor’s observation follows Gaussian distribution with mean 
0 and variance cr^, and the inter-sensor dependence is described by a Gaussian copula. Let ss denote the location 
of the signal source which is [0.75, 0.75] in this experiment. The variance of each sensor’s observation is inversely 
propotional to the distance between the sensor and the signal source, i.e., = l/Jsn — ss]. We set rEt = 1 and 

a = 4, thus, according to (|6]l, the largest coalition size that satisfies the energy efficiency constraint is [S'] =4. 

In the problem of estimation, the prior distribution of the unknown parameter 6 is assumed to have standard 
Gaussian distribution (zero mean, unit variance). The average FI of coalition S is given as 

I{S) = (28) 

where 1 is an all one vector with dimension [S'] by 1, and S 5 is the covariance matrix of coalition S, i.e., 
= [o'm,n]j„ with representing the covariance of sensor m and sensor n. In the detection problem, we 
set the parameters under hypothesis Hq and Hi to be 0o = 0 and 9i = y/2. The KLD corresponding to a coalition 
S is 


D{S) = (29) 

With the above setting, the average FI and KLD have exactly the same expression. Thus, we present the simulation 
results without distinguishing between the problems of estimation and detection. 

In the initialization step, each sensor is set to be a coalition by itself, i.e., S = {{!}, {2}, {3}, {4}, {5}, {6}, {7}, {8}}. 
By applying the proposed merge-and-split algorithm iteratively, three coalitions are formed as shown in Figure |3] 

It can be seen that each coalition contains physically apart, and thus statistically less dependent, sensors so that 
redundancy loss is avoided and diversity gain is taken advantage of to the largest degree. Also, the sensors closer 
to the signal source, who already have a good individual performance, form smaller coalitions, while the distantly 
located sensors form relatively large coalitions to improve their performance. Since a D^-stable solution is not 
guaranteed in this example, the resulting partition of the merge-and-split algorithm may change with different 
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initializations. With each iteration of merge-and-split, the overall payoff □ of sensors increases, until no further 
merge or split occurs as shown in Figure |4] 

We further consider a heterogeneous sensor network consisting of 28 sensors deployed in the same area of interest. 
We assume that observations of 14 sensors follow Gaussian distribution with 9 being the mean and unit variance, 
while observations of the other 14 sensors follow exponential distribution parameterized with 0. Within each Monte 
Carlo trial, the sensor locations are generated independently according to uniform distribution, through which the 
correlation matrix is obtained according to the dependence model in ( l27b . A student’s t copula parameterized by 
the correlation matrix with the degree of freedom = 4 is used to generate the dependence among sensors. 

'^We use “overall payofF’ to imply the payoff averaged over all sensors. The term “overall” will continually be used with the same implication 
in the later part of this section. 


• 
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Fig. 3. The deployment of the 8-sensor network and the final partition. 



Fig. 4. Average inference performance increases with each merge or split operation. 
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The performance corresponding to different coalition formation approaches for this particular sensor deployment 
is evaluated. A total of 100 Monte Carlo trials are conducted and the performance is averaged over these trials. 
We compare our proposed distributed algorithm based on coalition formation game with the approach of random 
coalition formation. In the random coalition formation method, a partition is randomly selected from the set of all 
partitions that satisfy the communication constraint with equality lf[ 

In the estimation problem, 0 is assumed to be standard Gaussian distributed. Figure |5] shows the overall estimation 
performance of our proposed distributed coalition formation approach, compared with the random coalition formation 

^The equality is to ensure a maximized inference performance, since the inference performance is nondecreasing in coalition size, according 
to Proposition [T] and Proposition!^ We make the coalition size to be exactly a, except for the one that may include less than a due to the fact 
that the total number of sensors N may not be an integer multiple of a. 



Fig. 5. Overall estimation performance vs. communication constraint a. 



Fig. 6. Overall actual communication cost vs. communication constraint a in estimation problem. 
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approach. As the constraint on communication cost gets looser {a increases), the overall estimation performance 
becomes better for both methods. However, since our approach fully explores and utilizes inter-sensor dependence 
during the coalition formation process, it achieves much better performance than the random selection method. 
The overall communication costs, defined as 1/|5| ^(>5'), are plotted in Figure |6] which demonstrates the 

superiority of our method in terms of communication efficiency. It has to be noted that, the average communication 
cost corresponding to our distributed coalition formation method is not the maximum communication cost that is 
allowed by the predefined constraint. It reflects the true cost of communication of the resulting partition, which 
may be much less than the maximum cost allowed. 

In the detection problem, we set 9q = 1 and 6 i = 2.4. Superior overall detection performance of the partitions 
resulting from our proposed coalition formation approach is shown in Figure |7] in comparison with the random 
coalition formation. The overall actual communication costs versus a are plotted in Figured] demonstrating a better 
communication efficiency of our approach. 

In our distributed algorithm, when two coalitions are unable to contribute much to each other in inference 
performance due to their dependence (or redundancy loss incurred), they will not merge into a new coalition. Thus, 
it forces the coalition to seek cooperation with other coalitions to which it can contribute more, or where it is highly 
valued due to the diverse information that it is able to bring in. In this way, the overall diversity gain is increased 
while redundancy loss is decreased. By formulating the distributed inference problem as a coalition formation game 
and solving the game using an iterative algorithm, we are able to obtain better system performance in terms of 
both inference performance and energy efficiency, compared with the random coalition formation scheme. 

In numerous practical scenarios, sensor networks are subject to changes. For example, sensors embedded in 
people’s cellphones change locations frequently. New sensors joining or existing sensors quitting also contributes 
to the time varying nature of the network. The distributed nature of our proposed coalition formation method in 
which sensors form coalitions automatically, makes it suitable for networks with time-varying configurations. 

VIII. CONCLUSION 

In this paper, we investigated a collaborative distributed inference problem in an energy constrained wireless 
sensor network with dependent observations. In the collaborative setting, sensors form coalitions and share obser¬ 
vations within the coalition for improved inference performance. We focused on the formation of non-overlapping 
collaborating coalitions such that each sensor’s performance is maximized while the energy constraint is satisfied. To 
analyze the benefit and cost of forming a certain coalition, we used copula theory to describe the dependence among 
observations, which provided “redundancy” and “diversity” aspects of inter-sensor dependence, respectively for the 
problem of estimation and detection. We defined GAFI and GKLD to quantify the diversity gain and redundancy 
loss in forming a coalition due to inter-sensor dependence. A coalition formation game was proposed for the 
distributed inference problem. A merge-and-split algorithm was utilized for our coalitional game and the stability 
of the outcome of our proposed algorithm was analyzed. Finally, numerical results were provided to demonstrate the 
performance of our game theoretical approach. Further investigation of the dependence-related concepts of diversity 
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gain and redundancy loss in inference problems under different scenarios is to be conducted in the future work. 
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